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PREFACE 


This  guidebook  was  prepared  by  System  Development  Corporation  under 
the  direction  of  the  Computer  Systems  Engineering  Directorate  of  the 
Electronic  Systems  Division  (ESD/TOI,  Formerly  MCI),  Air  Force  Systems 
Command.  The  Overview  guidebook  is  one  of  a series  of  Software  Acquisition 
Management  guidebooks  intended  to  help  ESD  Program  Office  personnel  in  the 
acquisition  of  embedded  software  for  command,  control  and  communications 
systems.  The  contents  of  the  guidebook  will  be  revised  periodically  to 
reflect  changes  in  software  acquisition  policies  and  practices  as  well  as 
feedback  from  guidebook  users. 


The  Software  Acquisition  Management  guidebook  series  is  currently  planned  to 
cover  the  following  topics  (National  Technical  Information  Service  accession 
numbers  for  those  already  published  are  shown  in  parentheses): 


) 


Regulations,  Specifications  and  Standards*  (AD-A016401) 

Contracting  for  Software  Acquisition  (AD-A020444) 

Monitoring  and  Reporting  Software  Development  Status  (AD-A016488) 
Statement  of  Work  Preparation  (AD-A035924) 

Reviews  and  Audits 

Computer  Program  Configuration  Management  (AD-A047308) 

Computer  Program  Development  Specification  (Requirements  Specification) 
Software  Documentation  Requirements  (AD-A027051  ) 

Verification  (AD-A048577) 

Validation  and  Certification 
Overview  of  the  SAM  Guidebooks 
Software  Maintenance 
Software  Quality  Assurance  (AD-A047318) 

Software  Cost  Estimation  and  Measurement 
Software  Development  and  Maintenance  Facilities  (AD-A038234) 

Life  Cycle  Events  ( AD-A0371 15) 
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SECTION  1 - INTRODUCTION 


1.1  PURPOSE 

The  Overview  guidebook  serves  as  the  introductory  volume  to  the  Command,  Con- 
trol, and  Communication  System  Software  Acquisition  Management  (SAM)  Guidebook 
series.  It  (1)  describes  the  series,  placing  it  in  the  overall  context  of 
the  Air  Force  Guidebook  program;  (2)  defines  the  intended  audience  of  the 
series;  (3)  tells  how  the  guidebooks  were  developed;  (4)  makes  recommendations 
concerning  guidebook  usage;  (5)  summarizes  the  contents  of  each  guidebook; 

(6)  provides  a subject  matter  index  for  the  series;  (7)  identifies  future 
guidebook  requirements;  and  (8)  provides  complete  bibliographical  citations 
for  each  guidebook  (see  Appendix  A). 


1.2  THE  COMMAND,  CONTROL,  AND  COMMUNICATION  SYSTEM  SAM  GUIDEBOOK  SERIES 

*3 

The  Coronand,  Control  and  Communications  (C  ) System  SAM  Guidebook  series  is 
one  of  three  such  series  (see  Figure  1)  designed  to  help  Air  Force  Program 
Office  (PO)  personnel  in  the  acquisition  and  management  of  embedded  software 
procured  under  Air  Force  800-series  regulations  and  related  concepts.  C3 
software  subsystems  can  be  characterized  as  follows: 

• Developed  and  maintained  in  an  evolutionary  environment. 

• Large,  customized,  and  unique  (normally  state-of-the-art). 

• On-line,  real-time,  closed-loop  operations. 

• Interactive  with  user  displays. 

• Employs  large  data  base  management  component. 

• Contains  significant  doctrinal  software  which  must  be  tailored 
to  the  user  and  usually  cannot  rely  on  off-the-shelf  packages. 

• Usually  uses  large,  ground-based  computers  (frequently  multi-site). 

• Capable  of  dealing  with  asynchronous,  event-driven  environments. 

t Services  extensive  automatic-communication  components  (often 
multi-site) . 

• Must  be  fault-tolerant  with  fail  safe/soft  and  recovery  attributes. 


Figure  1.  Air  Force  Guidebook  Program 
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The  C°  series  currently  consists  of  16  volumes  covering  various  aspects  of 
the  software  acquisition  process.  The  basic  purpose  of  the  guidebooks  is  to: 

• Communicate  preferred  procedures  for  planning,  scheduling, 
organizing,  directing,  and  evaluating  the  status  of  software 
development  programs. 

• Instruct  PO  personnel  in  the  implementation  of  relevant  DoD 
and  Air  Force  regulations,  specifications,  and  standards  (RSS). 

• Correlate  sources  of  information  concerning  management  of  software 
development,  acquisition,  operation,  and  support. 

• Provide  references,  explanations,  and  checklists  to  aid  PO 
personnel  in  the  early  detection  and  resolution  of  common 
SAM  problems. 

• Support  training  of  new  personnel . 

3 

Based  on  these  objectives  the  C guidebooks  emphasize  practical  approaches  to 
real-world  problems  within  the  context  of  Air  Force  and  DoD  RSS.  The  guide- 
books provide  Air  Force  PO  personnel  with  a comprehensive  review  of  software 
acquisition  management  practices  and  procedures.  They  provide  guidance  on 
how  to  assess  DoD  and  Air  Force  RSS;  how  to  determine  which  RSS  apply  to  a 
particular  aspect  of  software  acquisition;  and  how  to  resolve  conflicting  or 
overlapping  RSS  and  what  problems  may  arise  because  of  their  use.  Further, 
the  guidebooks  assist  PO  personnel  in  detecting  early  symptoms  of  common  SAM 
problems.  They  also  provide  advice  to  facilitate  communication  with  contractor 
and  other  PO  personnel . 

To  help  achieve  these  objectives,  guidance  is  presented  in  the  context  of  the 
system  acquisition  process.  The  relationship  between  the  guidebooks  and  the 
system  acquisition  life-cycle  establishes  a unifying  structure  over  the  entire 
series  (see  Figure  2).  This  approach  highlights  the  applicability  of  individual 
guidebooks  to  various  phases  of  the  life  cycle  and  emphasizes  the  relationship 
between  guidebooks. 

The  guidebooks  provide  three  general  classes  of  information: 

• Explanations  (lessons  learned,  common  pitfalls,  and  mistaken 
assumptions)  which  augment  official  guidance  with  useful 
experience.  This  kind  of  information  will  aid  the  guidebook 
user  in  answering  the  question:  Given  this  mass  of  formal 
definition  and  procedure,  how  do  I recognize  the  most  signi- 
ficant and  valuable  pieces  of  guidance,  how  can  I maneuver 
effectively  and  safely  among  sometimes  conflicting  sources  of 
guidance,  and  are  there  any  serious  hidden  implications  within 
these  sources? 
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Figure  2.  C Guidebooks  vs  System  Acquisition  Life  Cycle 


• Checklists  and  descriptions  of  proven  software  acquisition  manage- 
ment techniques  which  answer  the  questions:  What  are  the  early 
symptoms  of  common  SAM  problems  I might  encounter  in  daily 
operations,  and  what  tools  and  techniques  have  proven  effective 

in  the  past  in  monitoring  status,  in  identifying  problem  thresholds, 
and  in  re-creating  and  performing  post-mortems  on  management 
decisions? 

• References  and  index  lists  which  can  be  used  to  access  the  formal 
RSS  relevant  to  the  topic  presented.  These  aid  in  answering  the 
question:  What  are  my  official  management  opportunities  and 
constraints  in  dealing  with  this  particular  aspect  of  software 
acquisition? 

NOTE 

3 

Much  of  the  guidance  provided  in  the  C guidebook 
series  is  applicable  to  smaller  less  complex 
systems,  but  in  all  cases,  it  should  be  tailored 
to  the  needs  of  individual  projects. 


1.3  THE  INTENDED  GUIDEBOOK  AUDIENCE 

The  information  provided  in  the  C guidebook  series  is  directed  specifically 
to  Air  Force  PO  management  personnel  and  a member  of  the  Engineering  Division 
referred  to  as  the  Software  Director  (SD),  who  is  presumed  to  be  responsible 
for  managing  software  acquisition.  Although  the  SD  is  still  an  evolving 
position  without  specifically  defined  responsibilities,  the  C3  guidebook  series 
assumes  that  the  SD  has  the  Air  Force  management  experience  implied  by  the 
rank  of  Major.  The  series  assumes  that  his  systems  engineering  experience 
ranges  from  basic  to  highly  advanced. 


1 .4  HOW  THE  GUIDEBOOKS  WERE  DEVELOPED 
3 

The  C guidebook  series  was  developed  as  a cooperative  effort  between  the  Air 
Force  and  industry.  The  first  seven  guidebooks  were  developed  by  The  MITRE 
Corporation;  the  other  nine  were  developed  by  System  Development  Corporation. 
The  need  for  the  guidebooks  was  motivated  by  the  concerns  expressed  by 
potential  software  users,  by  reviewing  authorities,  and  by  software  designers 
and  implementors.  Studies  and  interviews  showed  dissatisfaction  with  software 
acquisition  management  in  several  areas: 

• Today's  RSS  are  purposely  written  in  a general  manner  which  allows 
an  experienced  SD  considerable  latitude  in  tailoring  the  acquisition 
to  the  specific  requirements  of  his  program. 
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• The  lack  of  experienced  SDs. 

• Standards  and  regulations  which  are  sometimes  unclear  or 
contradictory. 

• Known  differences  between  policy  and  practice. 

• SAM  principles  which  are  not  well  applied. 

3 

• Software  which  is  often  the  high  risk  item  of  C procurements. 

• Schedules  and  budgets  which  prove  to  be  unrealistic,  or  for  other 
reasons  are  not  met. 


These  and  related  problems  result  in  high  software  costs  and  unreliable  pro- 
ject schedules.  In  addition,  new  software  design  techniques,  such  as  struc- 
tured programming,  may  influence  the  ways  in  which  software  developments  are 
managed  in  the  future.  Although  a large  body  of  software  development  infor- 
mation is  available  in  written  form,  it  is  not  available  as  useful  guidance 
for  ready  reference  in  concise  or  focused  form.  Software  issues  are  often 
addressed  in  DoD  regulations  as  an  afterthought  in  areas  such  as  configuration 
management.  These  RSS  do  not  sufficiently  relate  to  software-unique  situa- 
tions and  are  not  focused  towards  the  everyday  problems  experienced  by  the 
Program  Manager.  Furthermore,  the  experience  of  varied  efforts  within  DoD  to 
acquire  defense  system  software  is  not  transferred  effectively  to  new 
personnel.  As  a result,  the  same  mistakes  tend  to  be  repeated  in  software 
acquisition  programs.  Thus,  there  is  a need  for  guidelines,  including  check- 
lists, examples,  and  other  how-to-do-it  information  for  the  day-to-day  use  of 
Program  Managers  and  their  staff. 

Further,  Software  Directors  and  Program  Managers  often  do  not  have  sufficient 
experience  or  the  training  to  adequately  recognize  and  solve  software  manage- 
ment problems.  Guidebooks  are  thus  needed  to  provide  a comprehensive,  concise 
review  of  software  acquisition  management  practices  and  procedures.  Although 
they  can't  assure  good  management,  their  conscientious  use  should  assure 
that  important  questions,  actions,  and  activities  won't  be  overlooked  or 
postponed  until  they  are  not  effective. 

3 

Planning  for  the  C guidebook  series  began  in  FY75,  using  the  sources  of  the 
ESD  Computer  Systems  Engineering  Directorate  (TOI*),  the  MITRE  Corporation, 
and  ESD  PO  experience.  The  first  seven  guidebooks  were  produced  by  the 
MITRE  Corporation  between  FY75  and  FY77: 


♦Formerly  MCI. 


• Regulations,  Specifications,  and  Standards 

• Contracting  for  Software  Acquisition 

• Statement  of  Work  Preparation 

• Software  Documentation  Requirements 

• Software  Development  and  Maintenance  Facilities 

• Life  Cycle  Events 


In  addition,  "A  Project  Guide  to  Content  Requirements  and  Audience  Needs," 
ESD-TR-75-308,  was  produced  by  MITRE  to  provide  guidance  for  the  development 
of  the  C^  guidebook  series.  It  covered  such  topics  as  terminology,  writing 
style,  depth  of  topical  coverage,  and  topical  scope. 

Subsequently,  a contract  was  let  to  System  Development  Corporation  to  produce 
the  rest  of  the  guidebooks  in  the  series.  SDC  was  tasked  to  review  applicable 
DoD,  Air  Force,  AFSC,  and  ESD  regulations,  manuals,  pamphlets,  MIL  specifica- 
tions, and  MIL  standards  to  compile  source  material  for  the  guidebooks. 
Annotated  outlines  for  each  of  the  guidebooks  were  produced  by  SDC  and 
reviewed  by  Air  Force*  and  MITRE  representatives.  Drafts  were  produced  for 
each  guidebook  with  substantial  review  inputs  from  Air  Force  PO  personnel, 
the  Air  Force  Contracting  Offices,  and  MITRE.  Face-to-face  reviews  of  each 
draft  were  conducted  with  the  guidebook  authors  to  maximize  both  Air  Force 
and  industry  input  as  well  as  to  assure  mutual  understanding.  Finally,  upon 
Air  Force  approval,  each  guidebook  was  produced  as  an  ESD  Technical  Report 
and  disseminated  to  the  field  through  Air  Force  channels  as  well  as  the 
facilities  of  the  National  Technical  Information  Service  (NTIS). 


1 .5  RECOMMENDED  USES  OF  THE  GUIDEBOOKS 
3 

The  C guidebook  series  relates  the  policy  expressec  in  existing  DoD  and  Air 
Force  RSS  to  current  ESD  and  industry  practices.  In  addition  to  providing 
the  PO  with  the  guidance  described  in  Paragraph  1.2,  the  guidebooks  provide 
an  interpretation  of  Air  Force  policy  regarding  software  acquisition  manage- 
ment. They  also  identify  criteria  which  PO  personnel  will  use  to  judge 
contractor  performance,  thus  providing  Air  Force-publ ishea  guidance  on 
software  acquisition  management  for  industry  use.  In  writing  the  guidebooks, 
the  following  specific  PO  needs  were  considered: 


including  AFSC,  AFLC,  ESD,  AFCS,  and  RADC. 
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• Detailed  guidance  for  specific  PO  activities. 

• A summary  of  the  latest  SAM  techniques. 

• Source  materials  for  SAM  training  courses. 

• Orientation  materials  and  references  for  new  PO  personnel. 


Perhaps  the  most  important  use  of  the  guidebooks  is  that  they  provide  a forum 
and  focus  for  continuing  innovation  and  improvement  of  the  SAM  process. 

1.6  AIR  FORCE  CONTRACT  MANAGEMENT  DIVISION  (AFCMD)  COMPUTER  SOFTWARE  POLICY 

Appendix  B of  this  guidebook  has  been  added  to  provide  guidance  concerning 
computer  software  policy  within  AFCMD.  This  discussion  is,  however,  more 
appropriate  to  the  Contracting  for  Software  Acquisition  guidebook  and  an 
expanded  discussion  of  this  subject  will  appear  in  a future  version  of  that 
guidebook. 
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SECTION  2 - INDIVIDUAL  C3  GUIDEBOOK  SUMMARIES 
2.1  INTRODUCTION 

This  section  presents  individual  summaries  of  the  following  guidebooks: 

• Regulations,  Specifications,  and  Standards  (see  2.2) 

• Contracting  for  Software  Acquisition  (see  2.3) 

• Monitoring  and  Reporting  Software  Development  Status  (see  2.4) 

• Statement  of  Work  Preparation  (see  2.5) 

• Reviews  and  Audits  (see  2.6) 

• Configuration  Management  (see  2.7) 

• Computer  Program  Development  Specification  (Requirements  Specification) 
(see  2.8) 

• Software  Documentation  Requirements  (see  2.9) 

• Verification  (see  2.10) 

• Validation  and  Certification  (see  2.11) 

• Software  Maintenance  (see  2.12) 

• Software  Quality  Assurance  (see  2.13) 

• Software  Cost  Estimation  and  Measurement  (see  2.14) 

• Software  Development  and  Maintenance  Facilities  (see  2.15) 

• Life  Cycle  Events  (see  2.16) 

Each  summary  highlights  the  basic  purpose  of  the  guidebook,  identifies  poten- 
tial uses,  and  describes  its  contents. 


2.2  REGULATIONS,  SPECIFICATIONS,  AND  STANDARDS* 

This  guidebook  serves  as  an  introduction  to  the  world  of  military  and  Govern- 
ment documents  pertaining  to  software  acquisition  management  and  development. 

It  begins  by  identifying  the  existing  types  of  official  documents  and  providing 
a table  of  guides,  lists,  catalogs,  and  indexes  to  the  various  forms  of  mili- 
tary and  Government  publications.  It  ends  with  two  indexes,  the  first  of 
which  lists  keywords  with  associated  military  and  Government  regulations, 
specifications,  and  standards  (RSS).  The  second  index  reverses  the  first  and 
lists  RSS  with  associated  key  words. 

The  RSS  guidebook  applies  to  software,  whether  or  not  it  is  acquired  as  an 
entity  or  as  a portion  of  a larger  system.  Therefore,  even  though  many  of 
the  documents  cited  do  not  specifically  refer  to  software  management  or 
development  tasks,  the  software  element  of  a system  assumes  the  same  measures 
of  management  control  and  development  quality  as  does  the  system.  Further, 

♦Reflects  the  contents  of  the  October  1975  version. 
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some  referenced  publications  deal  specifically  with  software  while  others 
apply  to  software  on  a broader  scale  (e.g.,  cost  control  systems,  or  WBSs). 

A major  section  of  the  RSS  guidebook  is  devoted  to  software  development  in 
the  system  life  cycle,  as  defined  by  Air  Force  800-series  regulations.  This 
section  identifies  a series  of  sequential  software  development  tasks  that  can 
be  applied  to  any  software  development  effort  regardless  of  the  system  life 
cycle  or  managment  controls  that  are  applied. 

A second  major  section  differentiates  between  the  types  of  programs  governed 
by  Air  Force  300-series  regulations  and  those  governed  by  Air  Force  800  series 
regulations.  This  section  provides  lists  of  300-series  and  800-series  regula- 
tions and  manuals  and  identifies  distinguishing  characteristics  between  the 
two  series. 

Two  other  major  sections  list  (1)  documents  pertaining  to  software  acquisition 
management  and  (2)  software  development  tasks,  while  an  appendix  presents 
abstracts  of  selected  software  acquisition  RSS. 

The  RSS  guidebook  was  revised  in  March  1978. 


2.3  CONTRACTING  FOR  SOFTWARE  ACQUISITION 

This  guidebook  provides  an  introduction  to  the  process  of  contracting  for  the 
development  of  computer  programs  acquired  under  Air  Force  800-series  regula- 
tions. It  starts  out  by  describing  the  Armed  Services  Procurement  Regulation 
(ASPR),  "the  bible  for  all  contracting  by  the  Department  of  Defense." 

A major  section  is  devoted  to  preliminary  procurement  planning,  including 
selection  of  the  basic  procurement  approach,  formal  procurement  planning,  and 
the  bid  package.  This  section  provides  a checklist  for  the  Software  Director 
to  use  in  planning  his  basic  procurement  approach  and  discusses  the  Advanced 
Procurement  Plan,  negotiated  procurements,  and  Determinations  and  Findings. 

Another  section  addresses  the  selection  of  a contractor,  including  contacts 
with  contractors,  RFPs,  proposal  evaluation,  contract  negotiation,  and  con- 
tract award. 

Still  another  section  discusses  contract  management.  It  summarizes  the 
responsibilities  of  the  Air  Force  Contracting  Officer,  the  Procuring  Contrac- 
ting Officer,  the  Administrative  Contracting  Officer,  the  Defense  Contract 
Administrative  Services,  the  Air  Force  Plant  Representative  Offices,  and  the 
Software  Director.  It  also  discusses  contract  changes  and  contract  termina- 
tion. 

The  last  section  lists  ASPR  references  that  are  directly  applicable  to  this 
guidebook.  It  also  abstracts  selected  RSS  of  particular  relevance  to  con- 
tracting for  software  acquisition. 

Finally,  an  appendix  provides  a list  of  source  selection  dos  and  don'ts. 
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2.4  MONITORING  AND  REPORTING  SOFTWARE  DEVELOPMENT  STATUS 

This  guidebook  provides  information  for  managers  and  technical  personnel 
engaged  in  the  planning,  monitoring,  and  reporting  of  the  technical  status 
of  a software  development  project.  Both  formal  procedures,  found  in  official 
regulations  and  manuals,  and  informal  methods,  from  military,  industrial,  and 
academic  experience,  are  discussed  to  provide  a concise  reference  for  the 
software  manager  to  assure  that  important  questions,  actions,  and  activities 
will  not  be  overlooked  or  postponed  until  they  are  no  longer  effective.  Using 
these  procedures  and  methods,  the  manager  should  be  able  to  identify  the  kinds 
of  information  relevant  to  his  project,  where  to  obtain  it,  how  to  use  it  to 
determine  status,  and  what  problems  may  be  associated  with  using  this  infor- 
mation. 

The  Monitoring  and  Reporting  Software  Development  Status  guidebook  is  relevant 
to  all  Air  Force  activities  responsible  for  planning,  developing,  and  acquiring 
computer  software  in  systems  managed  under  the  concepts  of  AFR  800-2.  A major 
section  of  the  guidebook  is  devoted  to  status-monitoring  tools  and  status 
reporting.  It  discusses  both  formal  and  informal  milestones,  periodic  status 
meetings,  contractor  and  Government  reports,  interviewing,  and  project  schedule 
representations . 

Another  section  discusses  contractual  planning  to  ensure  Government  visibility. 
It  addresses  program  management  planning  and  the  procurement  package,  includ- 
ing statement  of  work,  contract  data  requirements  list,  schedule,  and  indepen- 
dent monitoring  agencies. 

An  appendix  describes  new  software  methodologies  and  their  effect  on  manage- 
ment visibility.  The  discussion  addresses  top-down  and  bottom-up  imDlementa- 
tion  approaches,  modularity,  structured  design,  requirements  traceability, 
structured  programing,  and  functional  organization  of  personnel. 

Finally,  an  appendix  is  included  that  provides  summaries  of  selected  status 
factors  that  can  be  used  to  evaluate  project  progress.  Included  in  the 
appendix  are  discussions  of  documentation  quality,  stability  of  the  require- 
ments baseline,  interfaces,  programming  languages,  programing  practices  and 
standards,  project  complexity,  and  operating  systems.  In  addition,  the 
appendix  addresses  data  management,  personnel,  development  facility  quality, 
project  management,  and  non-subjective  data. 


2.5  STATEMENT  OF  WORK  PREPARATION 

This  guidebook  explains  the  preparation  of  a software-related  statement  of 
work  (SOW).  It  contains  a major  section  covering  planning  for  SOW  preparation 
and  then  presents  model  Full-Scale  Development  Phase  SOW  tasks. 

The  section  on  planning  heavily  emphasizes  the  use  of  work  breakdown  structures 
in  SOW  preparation  and  discusses  SOW-preparation  requirements,  general  sugges- 
tions for  SOW  preparation,  and  configuration  item  definition. 

The  section  on  model  SOW  tasks  includes  a table  of  contents  and  the  software- 
related  paragraphs  of  a hypothetical  Full-Scale  Development  Phase  SOW.  The 
SOW  presumes  to  prescribe  the  work  desired  from  a single  contractor  (at  the 
system  level)  to  develop  a postulated  one-of-a-kind  digital  communications 
message  switch.  The  SOW-prescribed  tasks  include  interfacing  the  system  with 
numerous  local  and  remote  digital  data  sources  and  sinks.  The  hypothetical 
planned  contract  covers  site  activation,  support  equipment,  and  administrative 
data,  as  well  as  software  acquisition,  computer  equipment  acquisition,  and 
system  engineering. 

Three  appendixes  are  included  in  the  guidebook.  The  first  addresses  work 
breakdown  structures,  which  it  defines,  describes  their  use,  and  summarizes 
the  various  types.  The  second  discusses  source  selection  plan  requirements. 

And  finally,  the  third  appendix  presents  extensive  guidance  regarding  the 
contents  of  RFPs. 


2.6  REVIEWS  AND  AUDITS 

This  document  provides  detailed  guidance  concerning  the  use  of  engineering 
design  reviews  and  configuration  management  audits  as  tools  to  monitor  a 
developing  organization's  technical  progress.  It  covers  the  following  formal 
reviews  and  audits: 

• System  Requirements  Review  (SRR) 

• System  Design  Review  (SDR) 

• Preliminary  Design  Review  (PDR) 

• Critical  Design  Review  (CDR) 

• Functional  Configuration  Audit  (FCA) 

• Physical  Configuration  Audit  (PCA) 

• Formal  Qualification  Review  (FQR) 
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Major  sections  are  devoted  to  general  requirements  for  reviews  and  audits, 
engineering  design  reviews,  and  configuration  management  audits.  Each  of 
these  sections  discusses  such  subjects  as  location  and  scheduling,  the 
responsibilities  of  participating  organizations,  and  the  conduct  of  reviews 
and  audits.  In  addition,  the  materials  to  be  reviewed  or  audited  are  listed 
and  suggested  evaluation  criteria  are  presented. 

Another  major  section  presents  modified  sample  forms  from  MIL-STD-1 521 A (USAF) 
which  can  be  used  to  identify  and  record  critical  data  during  reviews  and 
audi ts . 

Finally,  a section  on  the  more  common  reviews  and  audits  problem  areas  is 
included.  This  section  deals  with  responsibility  and  authority,  the  CPCI 
(Part  I)  Development  Specification,  and  the  scheduling  of  PDRs  and  CDRs. 


2.7  CONFIGURATION  MANAGEMENT 

This  guidebook  provides  basic  instructional  and  reference  materials  to  support 
the  application  of  Air  Force/DoD-prescribed  configuration  management  techniques. 
The  Configuration  Management  guidebook  is  one  of  the  largest  of  the  C3  series 
and  includes  seven  sections. 

The  first  section  presents  information  of  a background  and  introductory  nature, 
reviewing  general  concepts,  principles,  special  terms,  and  the  status  of  Air 
Force/DoD  configuration  management  standards. 

The  second  section  discusses  the  requirements  and  criteria  for  selecting 
assemblies  of  computer  program  code  to  be  identified  as  Computer  Program 
Configuration  Items  (CPCIs),  and  includes  a subsection  sunmarizing  the  sources 
and  coverage  of  standards  for  identification  numbers  and  markings. 

A third  section  is  devoted  to  specifications.  It  addresses:  specification 
types  and  forms;  the  specification  tree;  the  System  Specification;  Computer 
Program  Development  and  Product  Specifications;  other  types/forms  of  specifi- 
cations applicable  to  computer  programs;  and  comparisons  oetween  software  and 
hardware  with  respect  to  the  roles  of  their  specifications  in  the  system 
acquisition  cycle. 

The  fourth  section  covers  requirements  and  procedures  for  processing  changes 
to  approved  specifications.  It  identifies  organizational  factors,  explains 
change  classification,  describes  standard  forms,  and  discusses  procedures 
involved  in  the  preparation  and  processing  of  change  proposals.  It  includes 
a subsection  dealing  with  concepts  of  interface  control  and  the  documentation 
of  interfaces  involving  software. 
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Another  major  section  is  devoted  to  the  requirements  and  practices  of  document 
identification  and  maintenance  which  are  significant  to  configuration  manage- 
ment functions,  and  to  formal  reports/records  of  status  for  documents,  change  . 
proposals,  and  CPCIs. 

Still  another  section  addresses  factors  which  come  into  play  following  comple- 
tion of  development  and  initiation  of  the  CPCI  operations  at  a field  location. 
Using  a sample  System  Development  Test  and  Evaluation  situation  for  illustra- 
tion, it  identifies  the  nature  of  questions  to  be  anticipated  and  shows  how 
centralized  controls  and  procedures  described  in  the  preceding  sections  relate 
to  that  expanded  framework. 

The  last  section  contains  notes  written  in  response  to  questions  raised  by 
reviewers  of  a draft  version  of  this  guidebook,  and  pertaining  to  a few  of  the 
topics  covered  in  the  preceding  sections.  Its  coverage  includes:  computer 
programs  as  data  vs  Configuration  Items  ( CIs ) ; "software"  and  the  DoD 
directive;  system  segments;  problems  with  Chapter  2,  Volume  II,  of  AFR  800-14;  and 
Configuration  Index  questions. 


2.8  COMPUTER  PROGRAM  DEVELOPMENT  SPECIFICATION  (REQUIREMENTS  SPECIFICATION) 

This  guidebook  is  a detailed  development  guide  for  the  Computer  Program  Devel- 
opment (Part  I or  Type  85)  Specification.  It  provides  extensive  information 
on  both  the  contents  and  purpose  of  each  major  paragraph  of  a Development 
Specification.  It  describes  the  general  process  of  developing  technical 
requirements  and  outlines,  in  general,  the  roles  and  objectives  of  specifica- 
tions, placing  them  in  the  context  of  the  software  acquisition  life  cycle. 

A major  section  of  the  guidebook  is  devoted  to  general  guidelines  for  planning 
the  Development  Specification.  It  addresses:  the  specification  preparation 
team;  source  documents  for  format,  style,  and  classified  materials;  and 
summary  rules  for  general  empnasis  and  content.  This  section  also  provides 
detailed  guidance  for  Development  Specification  structure  and  outline. 

The  bulk  of  this  guidebook  consists  of  a detailed  paragraph-by-paragraph  dis- 
cussion of  the  required  contents  of  a Development  Specification.  It  is 
arranged  so  that  each  paragraph's  basic  content  and  purpose  is  defined, 
followed  by  a discussion  of  the  specific  types  of  information  to  be  provided. 


2.9  SOFTWARE  DOCUMENTATION  REQUIREMENTS 

This  guidebook  addresses  requirements  for  documentation  of  software  developed 
in  a large  system  acquisition,  including  the  documentation  associated  with 
its  development,  acquisition,  and  use.  It  emphasizes  the  determination  of 
documentation  needs,  the  preparation  of  the  Contract  Data  Requirements  List 
(CDRL) , and  the  specification  of  Data  Item  Descriptions  (DID)s. 
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The  bulk  of  this  guidebook  is  presented  in  four  sections.  A major  section 
addresses  software  documentation  requirements.  In  this  section,  the  author 
discusses  how  to  determine  these  requirements.  He  describes  the  purpose  of 
documentation  and  identifies  RSS  which  provide  guidance  on  the  determination 
of  documentation  requirements  for  software.  In  addition,  he  discusses  speci- 
fic factors  which  impact  documentation  requirements  and  presents  a series  of 
guidelines  that  can  be  used  to  determine  documentation  requi rements. 

Ifpther  major  section  describes  key  software  documents.  It  identifies  and 
describes  the  major  standard  Air  Force  data  items  which  apply  to  software, 
its  development,  its  acquisition,  and  its  use.  It  also  discusses  the  tailor- 
ing of  DIDs  to  meet  the  requirements  of  specific  programs.  In  addition,  it 
discusses  alterative  sets  of  software  documentation  (documents  used  by  DoD 
agencies  other  than  the  Air  Force)  that  may  be  applicable  to  some  programs. 

Still  another  section  of  the  guidebook  is  devoted  to  the  contractual  aspects 
of  data  item  acquisition.  This  section  addresses  contractual  specification 
of  the  documents  desired,  including  their  content,  format,  delivery  dates, 
numbers,  and  conditions  of  acceptance.  This  section  emphasizes  the  relation- 
ship among  the  SOW,  CDRL,  and  DIDs. 

J! 

The  final  section  of  the  guidebook  presents  general  conclusions  and  recommen- 
dations regarding  the  determination  of  documentation  requirements. 

A series  of  three  appendixes  provide  summary  material  on  data  items  relevant 
to  software  acquisition. 


VERIFICATION 


The  Verification  guidebook  provides  guidance  for  planning  and  managing  the 
implementation  of  software  verification  concepts  and  requirements  as  they 
relate  to  software  acquisition  management.  It  provides  a review  of  the 
verification  practices  and  procedures  employed  by  industry  and  set  forth  in 
relevant  RSS  and  describes  those  CPCI-oriented  system  enqineerinq  and  test 
activities  which  lead  to  verification. 


The  Verification  guidebook  starts  out  by  defining  the  term  verification  and 
distinguishing  it  from  the  terms  validation  and  certification.  It  then  goes 
on  to  a major  section  which  addresses  those  system  engineering  activities 
carried  out  to  ensure  that  the  CPCI  Development  Specifications  reflect  the 
requirements  allocated  from  the  System  Specification  (requirements  verifica- 
tion). 
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The  next  section  is  devoted  to  the  design  evaluation  activities  carried  out  to 
ensure  that  the  CPCI  design  continues  to  meet  the  requirements  of  the  Develop- 
ment Specification  as  the  design  proceeds  to  greater  levels  of  detail  (design 
verification). 

I 

The  last  section  is  concerned  with  the  informal  testing  performed  by  the  con- 
tractor, at  his  discretion,  to  assist  in  development,  provide  progress  visibi- 
lity, and  prepare  for  formal  testing  (computer  program  verification). 

An  appendix  describes  selected  commonly  used  support  tools  and  techniques  for 
computer  program  development  and  testing.  The  appendix  stresses  the  applica- 
bility of  these  aids  to  distinct  verification  tasks. 


2.11  VALIDATION  AND  CERTIFICATION 

This  guidebook  summarizes  the  software  acquisition  implications  of  validation 
and  certification.  It  begins  by  defining  validation  and  certification.  It 
then  goes  on  to  describe  those  system  engineering  activities  carried  out  to 
ensure  that  the  requirements  documented  in  the  System  Specification  accurately 
respond  to  the  operational  needs  called  for  in  the  Required  Operational 
Capability  (validating  the  System  Specification). 

The  next  section  addresses  those  Configuration  Item  (Cl)  integration  activities 
performed  to  assemble  and  check  out  previously  qualified  CIs  as  a fully 
functioning  system  (installation  and  checkout). 

Another  section  examines  the  software  aspects  of  system  validation  carried  out 
during  System  Development  Test  and  Operation  (DT&E)  and  Initial  Operational 
Test  and  Evaluation  (IOT&E)  to  demonstrate  that  the  completed  system  meets  the 
requirements  called  for  in  the  System  Specification  (validating  the  system). 

The  last  section  discusses  the  software-related  requirements  of  system  turn- 
over, transfer  of  management  responsibility,  and  system  certification. 


2.12  SOFTWARE  MAINTENANCE 

This  guidebook  addresses  those  acquisition  and  development  activities,  occur- 
ring throughout  the  SAM  cycle,  which  impact  software  maintenance.  It  includes 
discussions  of  system  turnover  to  the  using  command  and  the  transfer  of  pro- 
gram management  responsibility  to  the  supporting  command.  The  computer  pro- 
gram life  cycle  is  also  considered.  Most  of  the  information  provided  in  this 
guidebook  covers  the  implementing  command's  responsibilities  during  the  SAM 
cycle.  However,  software  maintenance  during  the  Deployment  Phase  is  also 
discussed  to  provide  the  background  for  proper  planning.  Current  programming 
concepts  are  discussed  as  well  as  RSS.  Within  these  constraints,  this 
guidebook  emphasizes  the  specification  and  procurement  of  maintainable  soft- 
ware, including  procurement  of  the  facilities,  support  tools,  and  documentation 
necessary  to  support  software  maintenance  activities. 
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Major  sections  are  devoted  to  the  acquisition  of  maintainable  software  and  to 
applicable  RSS.  The  former  addresses:  the  definition  and  specification  of 
maintainable  software;  monitoring  the  evolving  software  and  evaluating  con- 
tractor effectiveness;  design  change  and  error  correction  during  Subsystem 
DT&E;  transfer  and  turnover;  and  maintenance  during  the  Deployment  Phase.  The 
latter,  discusses  those  RSS  that  impact  software  maintenance.  As  used  in  this 
guidebook,  the  definition  of  software  maintenance  includes  the  ability  to 
modify  the  software.  This  section  therefore  relates  some  of  the  configuration 
management  RSS  to  software  maintenance. 

An  appendix  devoted  to  designing  maintainable  software  is  also  included  in  this 
guidebook.  It  describes  the  properties  of  maintainable  software,  including 
conceptual  organization,  modular  design,  self-monitoring  computer  programs, 
program  hooks  for  future  extensions,  and  design  methodology.  In  addition,  it 
covers  specific  techniques  that  facilitate  software  modification,  including 
computer  program  legibility,  parametric  organization,  stable  code,  and 
development  methodology,  e.g.,  structured  programming. 


2.13  SOFTWARE  QUALITY  ASSURANCE 

The  Software  Quality  Assurance  (QA)  guidebook  uses  the  definition  of  QA  expressed 
in  AFR  74-1,  i.e.,  "A  planned  and  systematic  pattern  of  all  actions  necessary 
to  provide  adequate  confidence  that  material,  data,  supplies,  and  services  con- 
form to  established  technical  requirements  and  achieve  satisfactory  performance." 
Within  this  context,  the  entire  concept  of  software  acquisition  management  is 
concerned  with  the  development  of  quality  software.  Special  attention  is  given 
to: 

• The  relationship  of  QA  to  the  other  acquisition  management 
disciplines. 

• The  integration  of  QA  requirements  within  the  system  acquisition 
process . 

• Contractual  aspects  of  QA. 

• Monitoring  the  implementation  of  QA  requirements. 

• Common  problems  and  proposed  solutions. 

• Pitfalls,  risk  areas,  and  danger  signals  as  they  occur  during  the 
System  Acquisition  Life  Cycle. 


/ 
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The  Quality  Assurance  guidebook  is  organized  into  three  sections  covering# 

Air  Force  Program  Office  (PO)'QA,  contractor  QA,  and  software  QA  at  ESD. 

The  first  section  relates  the  Air  Force  QA  program  to  the  major  mi  lestones  Af 
the  system  acquisition  cycle  as  they  occur  during  the  Conceptual,  Validation, 
and  Full-Scale  Development  Phases.  It  treats  objectives,  activities,  and  QA 
considerations  for  each  phase.  Discussions  are  supplemented  by  flow  chavts 
depicting  major  activities  within  each  phase. 

& 

The  second  section  provides  discussions  designed  to  assist  the  PO  in  evaluat- 
ing a contractor's  proposal  and  the  status  of  his  software  QA  program.  It 
discusses  software  QA  responsibilities,  necessary  activities  conducted  prior 
to  award  of  a Full-Scale  Development  Phase  contract,  and  contractor  QA  program 
implementation. 

The  last  section  describes  how  ESD  assists  its  POs  in  meeting  their  QA  require- 
ments. It  covers  the  evolving  QA  role  within  ESD  and  discusses  specific  QA 
aids . 

An  appendix  discusses  various  software  quality  issues.  It  begins  by  distin- 
guishing between  the  terms  software  QA  and  quality  software.  The  subjects 
covered  include  measuring  software  quality,  quality  vs  program  delays,  how 
much  QA  is  enough,  and  independent  support  contractors. 


2 . 1 4 SOFTWARE  COST  ESTIMATI ON  AND  MEASUREMENT 

This  guidebook  provides  a basic  understanding  of  the  current  methodologies 
used  in  the  formation  of  Air  Force  and  contractor  software  cos.t  estimates. 

It  provides  insight  into  some  of  the  problems  (and  reasons  for  the  problems) 
associated  with  software  cost  estimates  made  by  both  Government  and  industry. 
It  also  discusses  the  process  of  monitoring  software  costs  and  schedules 
while  providing  guidance  to  relevant  military  RSS  and  supporting  literature. 

In  this  guidebook,  careful  attention  is  paid  to  the  definition  of  cost  estima- 
tion terms.  The  guidebook  is  organized  so  as  to  introduce  the  reader  to  the 
global  concept  of  cost  analysis  and  proceed  from  there  to  more  specific  levels 
of  detail  directly  concerned  with  software  cost  estimation. 

A major  section  is  devoted  to  those  factors  associated  with  the  PO  that 
directly  contribute  to  the  formation  of  the  system  cost  estimate,  with  parti - 
cular  emphasis  on  the  software  element  portion  of  that  estimate.  The  topics 
covered  range  from  the  end  of  the  Conceptual  Phase  and  subsequent  program 
decision  to  submission  of  the  RFP  for  Full-Scale  Development. 
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Another  section  addresses  the  various  contractor  tasks  associated  with  prepar- 
ing a software  cost  proposal  in  response  to  an  RFP  for  Full-Scale  Development. 
This  section  presents  an  overview  of  the  numerous  software  cost  estimating 
techniques  used,  the  project-dependent  factors  that  impact  software  develop- 
ment costs  and  analyses,  and  task  specification  and  scheduling  with  regard  to 
the  WBS. 

The  last  section  discusses  the  evaluation  of  contractor  cost  proposals  in 
response  to  an  RFP.  It  also  discusses  issues  relating  to  the  requirements  and 
procedures  for  cost  reporting  and  performance  measurement  of  C3  system  acquisi- 
tion during  Full-Scale  Devefopment. 

An  appendix  summarizes  some  of  the  software  estimation  models  currently  used 
by  both  Government  and  industry. 

2.15  SOFTWARE  DEVELOPMENT  AND  MAINTENANCE  FACILITIES 

This  guidebook  examines  the  need  for  software  development  and  maintenance 
facilities,  discusses  policy  affecting  their  acquisition,  identifies  key 
management  decisions  and  technical  issues  involved  in  this  planning, 
surveys  existing  software  development  and  maintenance  facilities,  and 
identifies  potential  problems. 

A major  section  of  this  guidebook  is  devoted  to  explaining  the  roles  of  soft- 
ware development  and  maintenance  facilities  within  the  context  of  the  acquisi- 
tion and  computer  program  life  cycles. 

3 

Another  section  presents  the  results  of  a survey  of  14  C systems  to  determine 
what  types  of  development  and  maintenance  facilities  currently  exist  or  are 
planned.  It  identifies  characteristics  and  trends  and  summarizes  the  facilities 
of  each  system. 

Another  section  addresses  the  planning  end  acquisition  of  development  and 
maintenance  facilities.  It  discusses  the  management  decisions  and  technical 
issues  to  be  considered  and  identifies  applicable  RSSs. 

Finally,  a section  is  devoted  to  potential  problems  and  recommendations  for 
avoiding  them  or  minimizing  their  impact. 

Three  appendixes  are  included  in  this  document.  The  first  provides  brief 
descriptions  of  the  systems  surveyed.  The  second  provides  survey  data 
regarding  the  development  and  maintenance  facilities  planned  or  in  use  with 
the  systems  surveyed.  The  third  appendix  describes  representative  examples 
of  the  various  types  of  support  software  encountered  in  the  survey. 
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2.16  LIFE  CYCLE  EVENTS 

This  guidebook  explains  the  chief  activities,  events,  products,  and  software- 
related  efforts  that  normally  occur-  during  the  life  cycles  of  major  electronic 
systems  acquired  within  the  framework  of  the  800-series  of  Air  Force  regula- 
tions and  manuals.  The  800-series  normally  governs  acquisition  of  computers 
and  software  which  are  embedded  in  weapons  or  systems.  Some  of  this  soft-., 
ware  (e.g.,  application  programs)  may  be  built  expressly  for  the  weapons  or  (r 
system.  Some  (e.g.,  certain  operational  executives)  may  be  modified  versions 
of  off-the-shelf  software.  The  800-series  covers  the  research,  design,  develop- 
ment, engineering,  testing,  and  production  of  tactical  and  strategic  systems 
for  the  operational  inventory,  iri  contrast,  the  acquisition  of  off-the-shelf, 
commercially-marketed  data  processing  equipment  and  its  associated  support 
(non-functional)  software  for  business-like  applications  (e.g.,  payrolls, 
logistics,  personnel  records,  and  management  reporting)  is  normally  governed 
by  the  300-series  of  Air  Force  regulations  and  manuals.  This  guidebook  does 
not  address  acquisition  managed  in  accordance  with  the  300-series,  although 
some  of  its  principles  may  appiy. 

Two  potential  uses  are  anticipated  for  this  guidebook: 

• As  a tutorial  for  persons  relatively  inexperienced  in  the 
acquisition  of  large  systems  that  include  software. 

• As  a summary  of  material  relevant  to  software  acquisition  for 
those  otherwise  quite  familiar  with  the  acquisition  of  large 
systems . 

Major  sections  are  devoted  to  the  acquisition  life  cycle  including  an  overview 
of  the  life  cycle  and  one  section  each  for  the  Conceptual  Phase,  the  Validation 
Phase,  the  Full-Scale  Development  Phase,  and  the  Production  and  Deployment 
Phases.  Each  section  discusses  phase  objectives,  initiating  events,  primary 
activities  and  related  products,  and  terminating  events. 

Another  section  discusses  less  elaborate  acquisitions. 

The  last  section  summarizes  the  activities  and  products  of  the  computer  program 
life  cycle.  It  distinguishes  between  this  cycle  and  the  system  acquisition 
life  cycle  and  relates  the  two. 

An  appendix  summarizes  the  major  specification  provisions  affecting  software. 
This  appendix  should  be  compared  with  the  similar  discussions  in  the  Configura- 
tion Management  guidebook. 
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SECTION  3 - ESD  CRITICAL  ASSESSMENT  FACTORS  AND 
"SELECTED  SUBJECT  INDEXES' 


3.1  INTRODUCTION 

This  section  provides  indexes  to  the  guidebooks  for  the  36  Critical  Assessment 
Factors  (CAFs)  (see  Figure  3)  and  15  additional  selected  subjects  added  during 
the  development  of  this  guidebook  (see  Figure  4).  CAFs  are  defined  events  or 
products  that  are  essential  to  the  system  acquisition  process  and  indicate  the 
current  status  of  the  software  during  its  development  and  acquisition  cycles 
(for  further  discussion  of  CAFs,  see  4.4  of  the  Quality  Assurance  guidebook). 

In  using  the  indexes,  the  viewpoint  of  more  than  one  author  should  be  considered 
because  there  are  some  differences  in  definitions  and  in  guidance  from  guide- 
book to  guidebook.  In  addition,  all  discussions  tend  to  reflect  the  emphasis 
of  the  guidebook  in  which  they  appear  and  a particular  point  of  view  may  be 
unique  to  a specific  guidebook. 

Many  of  the  topics  included  in  the  indexes  were  discussed  extensively  in 
several  guidebooks  while  others  were  only  fleetingly  mentioned.  This  led  to 
differences  in  the  depth  of  discussion  referred  to  in  the  index.  For  example, 
the  Required  Operational  Capability  (ROC)  is  only  briefly  mentioned  in  the 
entire  C^  guidebook  series  whereas  the  Computer  Program  Development  SDecifi ca- 
tion has  an  entire  guidebook  devoted  to  it,  in  addition  to  extensive  discussions 
in  several  other  guidebooks. 


Please  note  that  the  Regulations,  Specifications,  and  Standards  guidebook  is 
not  covered  in  the  indexes.  It  was  purposely  left  out  because  indexes  to  the 
RSS  covering  the  51  subjects  of  Figures  3 and  4 are  beyond  the  scope  of  this 
guidebook. 
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CAF 

GUIDEBOOK 

TEXT  REFERENCE 



Required  Operational 
Capability 

Configuration  Management 

Computer  Program  Develop- 
ment Specification 

3.2.2,. 

See  Figure  1-2. 

Veri fi cation 

1.2.2,  Figure  3. 

Validation  & Certification 

1.2.1,  2.2,  Figure  1 . 

Software  Quality  Assurance 

2. 2. 2.1,  2. 2. 2. 1.1, 
Figure  2. 

Life  Cycle  Events 

3.2,  Table  1. 

Costing/Sizing 

Statement  of  Work 
Preparation 

See  Entire  Volume. 

Software  Documentation 
Requirements 

See  Section  II 
(Page  21). 

Software  Cost  Estimation 
& Measurement 

See  Entire  Volume. 

Program  Management 

Di recti ve 

Monitoring  & Reporting 
Software  Development 

Status 

3.1,  3.2. 

Validation  & Certi fication 

4.2,  5.1. 

Software  Maintenance 

3.1,  3.2. 

Software  Quality  Assurance 

2.2.1,  2. 2. 2.1, 

2. 2. 2. 1.2,  4.5. 

Software  Development 
& Maintenance  Facilities 

4.1.2,  4.4,  5. 

Life  Cycle  Events 

See  Tables  1 , 2,  4 3, 
and  2,  3.2,  4.2,  4.4, 
4.4.1,  4.4.3,  5.2 

Figure  3.  ESD  CAF  Index 
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Program  Management  Plan 

Monitoring  & Reporting 
Software  Development  Status 

3.2. 

Statement  of  Work 
Preparation 

2,  2.2. 

Software  Documentation 

Requi rements 

| 

See  Section  II  (Page 

23) , and  Section  IV 
(Page  47). 

Validation  & Certification 

4.2,  4.2.1,  4.2.3. 

Software  Maintenance 

3.1,  3.2. 

Software  Quality  Assurance 

1 

2.1 , 2. 2. 2. 1.2,  2. 2. 2. 2, 
2. 2. 2. 2.1,  2. 2. 2. 2. 2, 
4.5. 

Software  Cost  Estimation 
& Measurement 

2.1. 

Software  Development 
& Maintenance  Facilities 

4.1.2,  4.4. 

Life  Cycle  Events 

See-Tables  1 , 2,  & 3, 
and  4.4,  4.4.1  , 4.4.3. 

Procurement  Plan 

Contracting  for  Software 
Acquisition 

2 thru  2.4. 

Statement  of  Work 
Preparation 

2. 

Software  Quality  Assurance 

2. 2. 2. 2.1 , 2. 2. 2. 3. 2, 
4.5. 

Software  Cost  Estimation 
& Measurement 

1.2. 

Life  Cycle  Events 

See  Tables  1 , 2,  & 3. 

Figure  3.  ESD  CAF  Index  (cont'd) 
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TEXT  REFERENCE 

System  Specification 

Monitoring  & Reporting 
Software  Development 

Status 

1.5,  2.2,  2.2.1. 

Statement  of  Work 
Preparation 

2.2,  2.3,  Exhibit  1, 
Appendix  C. 

Reviews  & Audits 

See  Entire  Volume. 

Configuration  Management 

2.2,  Section  3,  4.3.1 , 
Section  5,  7.4. 

Computer  Program  Develop- 
ment Specification 

1.2,  1.3.1. 

Software  Documentation 

Requ  i remer.ts 

See  Tables  1 & 2, 
Section  III  (Pages 

56  & 108),  Appendix  I 
(Page  140). 

Validation  & Certification 

2.1,  2.2,  4.2.3. 

Software  Maintenance 

See  Figure  1. 

Softv'are  Quality  Assurance 

2.2.1,  2. 2. 2. 2. 2, 

2. 2. 2. 3 thru  2. 2. 2. 3. 2, 

2. 3. 1.1. 2,  2. 3. 1.3, 

2.3.1. 3. 2,  4.5. 

Software  Cost  Estimation 
& Measurement 

See  Figure  1 . 

Life  Cycle  Events 

Tables  1 thru  4, 

3.3,  4.3,  Appendix  A. 

Computer  Resources 

Integrated  Support  Plan 

Contracting  for  Software 
Acquisition 

3.2. 

Monitoring  & Reporting 
Software  Development  Status 

3.2. 

Statement  of  Work 
Preparation 

2 

Configuration  Management 

2.1.2,  4.3. 

Software  Documentation 
Requirements 

See  Sections  II  (Page 
23)  & III  (Pages  26, 

39,  46,  47)  and 

Tables  I,  VI,  & VII. 

Validation  & Certification 

5.1,  5.2. 

Figure  3.  ESD  CAF  Index  (cont’d) 
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Computer  Resources 

Integrated  Support  Plan 
(cont'd) 

Software  Maintenance 

1.2,  1.3.2,  2.1, 
2.1.1,  2.4,  2.4.1, 
2.5,  3.1,  3.2. 

Software  Quality  Assurance 

2. 2. 2. 2. 2,  2.3, 

2. 3. 1.3,  2. 3. 1.3.1, 

4.1.2,  4.2,  4.4,  4.5. 

Life  Cycle  Events 

See  Tables  2 & 3 and 
4.4,  4.4.2,  6.4. 

Statement  of  Work 

Contracting  for  Software 
Acquisi tion 

2.4. 

Monitoring  & Reporting 
Software  Development  Status 

2.2.1,  3.3,  3.3.1. 

Statement  of  Work  Prepara- 
tion. 

See  Entire  Volume. 

Software  Quality  Assurance 

2. 2. 2. 3. 2,  3.3,  4.5. 

Software  Cost  Estimation 
& Measurement 

3.3.1,  4.2. 

Software  Development  & 
Maintenance  Facilities 

4.5. 

Life  Cycle  Events 

1 . 

Contract  Data  Requirements 
List 

Contracting  for  Software 
Acquisition 

2.4. 

Monitoring  & Reporting 
Software  Development  Status 

2.2.1,  2.4,  3.3, 

3.3.1,  3.3.2. 

Statement  of  Work  Prepara- 
tion 

1,  1.2,  2,  2.1.5, 
2.1.6,  2.2,  Exhibit  1 , 
Appendix  C. 

Reviews  & Audits 

4.2.2. 

Configuration  Management 

7.1. 

Figure  3.  ESD  CAF  Index  (cont'd) 
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Contract  Data  Requirements 
List  (cont'd) 

Software  Documentation 
Requirements 

See  Sections  II 
(Pages  15,  16,  20,  & 
21)  & IV  (Pages  ISO- 
132). 

Software  Maintenance 

2.1,  2.1.2,  3.1,  3.2. 

i 

Software  Quality  Assurance 

3.3,  4.5. 

Software  Cost  Estimation 
& Measurement 

4.2. 

Software  Development 
& Maintenance  Facilities 

4. 3. 1.5,  4.5. 

Source  Selection  Plan 

Contracting  for  Software 
Acquisition 

3.0,  3.3. 

Statement  of  Work 
Preparation 

See  Appendix  B. 

Software  Quality  Assurance 

4.5. 

Source  Selection 

Contracting  for  Software 
Acquisition 

j 

3.0,  3.1 , 3.3,  3.4, 

5.0,  Appendix. 

Statement  of  Work 
Preparation 

See  Appendix  B. 

Software  Quality  Assurance 

2. 3. 1.3,  4.3,  4.5. 

Software  Cost  Estimation 
& Measurement 

See  Section  4. 

Life  Cycle  Events 

See  Tables  1 & 2. 

Contract 

Contracting  for  Software 
Acquisition 

9 9 

Monitoring  & Reporting 
Software  Development  Status 

See  Section  3. 

Statement  of  Work  Prepara- 
tion 

See  Section  1 & > 
Appendix  C. 

Figure  3.  ESI)  CAF  Index  (cont'd) 
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Contract  (cont'd) 

[Software  Documentation 

Requirements 

See  Section  IV  (Pages 
130-132). 

Software  Quality  Assurance 

2. 4. 1.1,  2. 4. 1.1.1, 
3.3,  3.4,  4.5. 

Software  Cost  Estimation 
& Measurement 

2.3. 

Software  Development  & 
Maintenance  Facilities 

4.5. 

Life  Cycle  Events 

4.3.1 .3,  4.4,  Tables 

2 & 3,  Appendix  A. 

Computer  Program  Development 
Plan 

Contracting  for  Software 
Acquisi tion 

See  Appendix. 

Monitoring  & Reporting 
Software  Development  Status 

3.2,  3.3.1. 

Statement  of  Work  Prepara- 
tion 

2.1.7,  Exhibi t 1 , 
Appendix  C. 

Reviews  & Audits 

3.3.2. 

Computer  Program  Develop- 
ment Specification 

1.3.2c. 

Software  Documentation 
Requirements 

See  Section  III 
(Pages  31-40,  51-53). 

Verification 

2.2.1,  2.3,  2.3.1. 

Software  Maintenance 

1.3.2,  2.2.2,  2.2.3. 

Software  Quality  Assurance 

2. 3. 1.3,  2. 3. 1.3. 2, 

2. 3. 2.1,  3.3,  3.4, 
3.4.4,  3.4.9,  3.4.10, 
3.4.11. 

Software  Cost  Estimation 
& Measurement 

3.3,  4.2. 

Software  Development 
& Maintenance  Facilities 

4.1.2,  4.2,  4.4. 

Life  Cycle  Events 

See  Tables  2 & 3 
and  4.4.5. 

Figure  3.  ESD  CAF  Index  (cont'd) 
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TEXT  REFERENCE 


CPC  I Structure 

Statement  of  Work 

Prepa ration 

: 1 

23,3,  Appendix  A. 

Configuration  Management 

See  Section  2. 

Life  Cycle  Events 

4. 3. 1.3. 

Configuration  Management 

Plan 

Monitoring  & Reporting 
Software  Development  Statu; 

3.2,  3.3.1. 

Statement  of  Work  Prepara- 
tion 

See  Exhibit  1 . 

Configuration  Management 

See  Section  4. 

Software  Documentation 
Requirements 

See  Section  III 
(Page  54)  & Appendix 

I (Page  139). 

Software  Quality 

Assure  nee 

3.4.3,  3. 4.1.1. 

Configuration  Control 

Contracting  for  Software 
Acquisition. 

A. 2. 

Statement  of  Work  Prepara- 
tion 

2.3,  Exhibit  1 . 

Configuration  Management 

See  Entire  Volume 
(especially  Sections 

4 & 6). 

Software  Maintenance 

2.4.1 , 2.4.2. 

Software  Development  & 
Maintenance  Facilities 

4.2. 

Life  Cycle  Events 

4.3.1 .2. 

Figure  3.  ESD  CAF  Index  (cont'd) 
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System  Requirements  Review 

Reviews  & Audits 

See  Entire  Volume. 

Computer  Program  Develop- 
ment Specification 

1.3.2. 

Veri fi cation 

2.2.2,  4.1.1. 

Validation  & Certification 

2.2. 

Software  Quality  Assurance 

2. 3. 1.1,  2. 3. 1.1. 2. 

Life  Cycle  Events 

See  Table  2. 

System  Design  Review 

Monitoring  & Reporting 
Software  Development  Status 

2.2,  2.2.1,  3.3.1. 

• 

Reviews  & Audits 

See  Entire  Volume. 

Computer  Program  Develop- 
ment Specification 

1.3.2. 

Verification 

1.2.1,  2.2.3. 

Validation  & Certification 

2.2. 

Software  Maintenance 

See  Figure  1 . 

Software  Quality  Assurance 

2. 3. 1.2,  2. 3. 1.2. 2. 

Software  Cost  Estimation 
& Measurement 

See  Figure  1 . 

Life  Cycle  Events 

See  5.3  and  Tables 

2 & 3. 

Development  Specification 

Monitoring  & Reporting 
Software  Development  Status 

1.5,  2.2,  2.2.1. 

Reviews  & Audits 

See  Entire  Volume. 

Configuration  Management 

2.2,  Section  3,  4.3.1 , 
4.4,  4.5.2,  Section  5. 

Computer  Program  Develop- 
ment Specification 

See  Entire  Volume. 

Figure  3.  ESD  CAF  Index  (cont'd) 
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Development  Specification 
(cont'd) 


Software  Documentation 
Requi remen ts 

Verification 

Software  Maintenance 

Software  Quality 
Assurance 


See  Table  I and 
Section  III  (Page  62). 

2.3.2. 

See  Figure  1 , 1.3.2, 

2.1.2. 

2 2. 2.2. 1,  2.3, 

2. 3. 1.3,  2. 3. 1.3. 2, 
2.3.2,  2. 3. 2.1,  2.4 
thru  2.4.1 .1 .2, 

2. 4. 1.4.1,  2.4.2. 


Life  Cycle  Events  See  Table  2,  3,  & 4, 

and  4. 3. 1.1,  4. 3. 1.2, 
Appendix  A. 


Figure  3.  ESD  CAF  Index  (cont'd) 
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Training  Plan 

Software  Documentation 
Requirements 

See  Table  II,  Section 

III  (Page  55), 

Appendix  I (Page  144). 

, 

Validation  & Certification 

5.2,  5.3. 

Life  Cycle  Events 

See  4.4,  4.4.4  and 

Tables  2 & 3. 

Test  Plan 

Monitoring  & Reporting 
Software  Development  Status 

2.2,  3.3.1. 

Reviews  & Audits 

3. 1.2. 2,  3.2.1. 

Computer  Program  Develop- 
ment 

3.19,  3.19.1. 

Software  Documentation 
Requirements 

See  Tables  I & II, 
Section  III  (Pages 

94,  97,  113),  Appen- 
dix I (Page  148). 

Veri  fi  cation 

2.2.1,  2.3.3. 

Validation  & Certification 

3.2,  4.2.4,  5.1. 

Software  Maintenance 

2.2.1. 

Software  Quality  Assurance 

2. 2. 2. 3,  2.3,  2. 3. 1.2.1, 

2. 3. 1.3. 2,  2. 4. 1.1. 2,  1 

2. 4. 1.2,  3.3,  3.4.4. 

Life  Cycle  Events 

Tables  1,  2,  3,  & 4 
and  4.3.1 .2,  4.4, 

4.4.3. 

Preliminary  Design  Review 

Monitoring  & Reporting 
Software  Development  Status 

2.2,  2.2.1,  3.3.1. 

Reviews  & Audits 

See  Entire  Volume. 

Computer  Program  Develop- 
ment Specification 

1.3.2. 

Veri fi cation 

1.2.1,  3.2.1.  v 

Figure  3.  ESD  CAF  Index  (cont'd) 
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Preliminary  Design  Review 
(cont'd) 

Software  Maintenance 

Software  Quality  Assurance 

See  2.2.1  and  Figure  1 . 

2. 4. 1.1,  2. 4. 1.1. 2, 

2. 4. 1.2.1,  2. 4. 1.4. 2. 

Software  Cost  Estimation 

4 Measurement 

See  Figure  1. 

[ 

Software  Development  4 
Maintenance  Facilities 

2.4,  4.6. 

Life  Cycle  Events 

Tables  344  and  5.3. 

Interim  Progress  Reviews 

Monitoring  & Reporting 
Software  Development  Status 

2.3,  2.3.1,  2.3.2, 

3.3.1. 

Product  Specification 

Monitoring  4 Reporting 
Software  Development  Status 

1.5,  2.2,  2.2.1. 

Reviews  4 Audits 

See  Entire  Volume. 

Configuration  Management 

See  Section  3,  4.3.1 , 
4.4,  4.5.2,  Section  5. 

Computer  Program  Develop- 
ment Specification 

1 .2. 

Software  Documentation 

Requ: rements 

See  Table  I 4 Section 

III  (Page  66). 

Software  Maintenance 

See  Figure  1 , 1.3.2. 

Software  Quality  Assurance 

2.4,  2. 4. 1.4,  2. 4. 1.4.1, 
2. 4. 1.4. 2,  2.4.2, 

2. 4. 2.1. 

Life  Cycle  Events 

See  Tables  344, 

5.3,  Appendix  A. 

Test  Procedures 

Monitoring  4 Reporting 
Software  Development  Status 

?.?.  3.3.1 . 

Computer  Program  Develop- 
ment Specification 

3.19,  3.19.1. 

Figure  3.  ESD  CAF  Index  (cont'd) 
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GUIDEBOOK 

TEXT  REFERENCE 

Test  Procedures  (cont'd) 

Software  Documentation 
Requirements 

See  Tables  I & II, 
Section  III  (Pages 

94,  97). 

Verifi  cation 

3.2.3. 

Validation  & Certification 

4.2.4. 

Life  Cycle  Events 

See  Tables  3 & 4, 

4. 3. 1.2. 

Critical  Design  Review 

Monitoring  & Reporting 
Software  Development  Status 

2.2,  2.2.1,  3.3.1 . 

Reviews  & Audits 

See  Entire  Volume. 

Veri  fi  cation 

1.2.1,  3.2.2. 

Software  Maintenance 

See  Figure  1,  2.2.2. 

Software  Quality  Assurance 

2. 4. 1.2,  2. 4. 1.2. 2. 

Software  Cost  Estimation 
& Measurement 

See  Figure  1 . 

Software  Development 
& Maintenance  Facilities 

2.2,  4.6. 

1 

Life  Cycle  Events 

See  Tables  3 & 4, 

5.3. 

Coding 

Monitoring  & Reporting 
Software  Development  Status 

See  Appendix  III 
(5  thru  5.2.5). 

Veri  fi  cation 

See  Appendix  A (3.1 
thru  3.1.4,  3.2  thru 
3.2.2). 

Software  Maintenance 

See  Figure  1 , 2.2.3, 
Appendix  A (Section 

a X 

j;. 

Software  Quality  Assurance 

2. 4. 1.3. 

Software  Cost  Estimation 
& Measurement 

See  Figure  1. 

L 

Figure  3.  ESD  CAF  Index  (cont'd) 
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TEXT  REFERENCE 

Coding  (cont'd) 

Software  Development 
& Maintenance  Facility 

2.2. 

Life  Cycle  Events 

See  Table  4. 

Preliminary  Qualification 
Tests 

Monitoring  & Reporting 
Software  Development  Status 

2.2,  2.2.1. 

Computer  Program  Develop- 
ment Specification 

3.20. 

Verification 

1.2.1,  4.1,  4.1.3, 

4.2,  4.2.1. 

Software  Quality  Assurance 

2. 4. 1.3  thru  2. 4. 1.3. 2, 
2. 4. 1.4.1,  2. 4. 1.4. 2, 

2. 4. 1.2. 

Software  Development 
& Maintenance  Facility  •» 

2.2. 

Life  Cycle  Events 

See  Tables  3 & 4. 

Formal  Qualification 

Tests 

Monitoring  & Reporting 
Software  Development  Status 

2.2,  2.2.1. 

Reviews  & Audits 

2.1 . 

Computer  Program  Develop- 
ment Specification 

3.2.1. 

Verification 

1.2.1,  4.1,  4.1.3, 

4.2,  4.2.2. 

Software  Maintenance 

2.2.4. 

Software  Quality  Assurance 

2. 4. 1.3.1,  2. 4. 1.4, 

2. 4. 1.4.1. 

Software  Cost  Estimation 
& Measurement 

See  Figure  1. 

Software  Development  & 
Maintenance  Facilities 

2.2. 

Life  Cycle  Events 

Tables  3 & 4,  5.3. 

Figura  3.  ESD  CAF  Index  (cont'd) 
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Functional  Configuration 
Audit 

Monitoring  & Reporting 
Software  Development  Status 

22. 

Reviews  & Audits 

See  Entire  Volume. 

Veri  fi  cation 

1.2.1. 

Software  Maintenance 

2.2.5. 

Software  Quality  Assurance 

2. 4. 1.4,  2. 4. 1.4. 2. 

Life  Cycle  Events 

See  Table  3,  5.3. 

Users  Manuals 

Software  Documentation 

Requi rements 

See  Table  I and 

Section  III  (Pages 

81,  110). 

Life  Cycle  Events 

See  Table  3. 

Physical  Configuration 

Audit 

Monitoring  & Reporting 
Software  Development  Status 

2.2. 

Reviews  & Audits 

See  Entire  Volume. 

Software  Maintenance 

2.2.6. 

Software  Quality  Assurance 

2. 4. 1.4,  2. 4. 1.4. 2. 

Life  Cycle  Events 

See  Table  3,  5.3. 

System/ Integration  Tests 

Monitoring  & Reporting 
Software  Development  Status 

1.6,  2.2.1. 

Reviews  & Audits 

3. 1.2. 2. 

Configuration  Management 

See  Section  6. 

Computer  Program  Develop- 
ment Specification 

3.2.2. 

Software  Documentation 
Requirements 

See  Tables  I & II, 
Section  III  (Pages 

54,  97,  104), 

Appendix  I (Page  "48). 

Veri fication 

1.2.2. 

Figure  3.  ESD  CAF  Index  (cont'd) 
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System/ Integration  Tests 
(cont'd) 


GUIDEBOOK 


Validation  & Certification 
Software  Maintenance 


TEXT  REFERENCE 


.2,  3.1 , 3.2, 
section  4,  5.1 , 5.2. 

see  Figure  1 . 


Software  Quality  Assurance  2. 4. 1.3.1. 

Software  Cost  Estimation  See  Figure  1. 
and  Measurement 

Software  Development  & 

Maintenance  Facilities 


Life  Cycle  Events 


ISee  Tables  3 & 4, 


Formal  Qualification  Review  Monitoring  & Reporting  2.2. 

Software  Development  Status 

Reviews  & Audits 


Verification 
Life  Cycle  Events 


Transi tion/Turnover 
Agreement 


Validation  & Certification  See  Section  5. 


Software  Maintenance 


Life  Cycle  Events 


1.2,  2.4  thru  2.4.3, 

3.1. 

4.2,  6.4. 


Figure  3.  ESD  CAF  Index  (cont'd) 
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SELECTED  SUBJECT 


GUIDEBOOK 


I EXT  REFERENCE 


Structured  Programming 

Monitoring  & Reporting 
Software  Development  Status 

See  Appendix  III 
(Section  5) . 

Software  Maintenance 

See  3.4  4 Figure  9. 

Trade/Design  Studies 

Software  Documentation 
Requirements 

See  Section  III 
(Page  92). 

Software  Quality  Assurance 

2. 2. 2. 3. 2,  2. 3. 1.1. 2. 

Software  Development  & 
Maintenance  Facilities 

2.2. 

Work  Breakdown  Structure 

Monitoring  & Reporting 
Software  Development  Status 

3.3.1. 

Statement  of  Work  Prepara- 
tion 

1 , 1.4,  2,  2.1.1, 
2.1.3,  2.2,  2.3,  3, 
3.2,  Exhibit  1 , 
Appendix  A. 

Software  Maintenance 

2.1.3. 

Software  Cost  Estimation 
& Measurement 

2.1.1,  2.1.2,  3.3, 

3.3.1,  3.3.2,  4.3. 

Life  Cycle  Events 

See  Table  1 & 3.3, 
4.3.1 .2. 

Schedule 

Contracting  for  Software 
Acquisition 

2.4. 

Monitoring  & Reporting 
Software  Development  Status 

2.7,  3.3,  3.3.1, 

3.3.3. 

Statement  of  Work  Prepara- 
tion 

1. 

Software  Documentation 
Requirements 

See  Section  II  (Page 
20)  & Appendix  I 1 

(Page  137). 

Software  Quality  Assurance 

2. 4. 1.4. 2. 

Figure  4. 

Selected  Subject  Index 
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SELECTEO  SUBJECT 

GUIDEBOOK 

TEXT  REFERENCE 

Schedule  (cont'd) 

Software  Cost  Estimation 
& Measurement 

3.2.7,  3. 2. 7.1,  3.3, 
3.3.2,  4.2. 

Software  Development  & 
Maintenance  Facilities 

4. 3. 1.5. 

Verification 

Verification 

See  Entire  Volume. 

Validation  & Certification 

Validation  & Certification 

See  Entire  Volume. 

Conceptual  Phase  Activities 

Computer  Program  Develop- 
ment Specification 

1.3.1. 

Validation  & Certificatior 

See  Section  2. 

Software  Maintenance 

1.2. 

Software  Quality  Assurance 

2.2  thru  2. 2. 3.1. 

Software  Cost  Estimation 
& Measurement 

See  Figure  1 . 

Software  Development  & 
Maintenance  Facilities 

2.1. 

Life  Cycle  Events 

See  Section  3. 

Validation  Phase  Activities 

Reviews  & Audits 

1.5,  2.4,  3.1  thru 
3.2.3. 

Computer  Program  Develop- 
ment Specification 

1.3.2. 

Verification 

1.2,  2.1  thru  2.2.3. 

Validation  & Certification 

See  Section  2. 

Software  Maintenance 

1.2,  2.1.2. 

Software  Quality  Assurance 

2.3  thru  2. 3. 2.1. 

Figure  4.  Selected  Subject  Index  (cont'd) 
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SELECTED  SUBJECT 
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TEXT  REFERENCE 

Validation  Phase  Activities 
(cont'd) 

Software  Cost  Estimation 
& Measurement 

See  Figure  1 . 

Software  Development  & 
Maintenance  Facilities 

2.1. 

Life  Cycle  Events 

See  Section  4. 

Full-Scale  Development 

Phase  Activities 

Statement  of  Work  Prepara- 
tion 

See  Section  3. 

Reviews  & Audits 

2.4,  3.3  thru  3.4.3, 
Section  4. 

Verification 

2.3  thru  2.3.3, 
Section  3,  Section  4. 

Validation  & Certification 

See  Sections  3 & 4. 

Software  Maintenance 

1.2,  2.1.2,  2.2.  thru 
2.2.6,  2.3  thru  2.4.3. 

Software  Quality  Assurance 

2.4  thru  2. 4. 2.1. 

Software  Cost  Estimation 
& Measurement 

See  Fi gure  1 . 

Software  Development  & 
Maintenance  Facilities 

2.1 . 

Life  Cycle  Events 

See  Section  5. 

Deployment  (Operational/ 
Support)  Phase  Activities 

Validation  & Certification 

Software  Maintenance 

See  Section  5. 

1.2,  2.3,  2.5. 

Software  Cost  Estimation 
& Measurement 

See  Figure  1 . 

Software  Development  & 
Maintenance  Facilities 

2.1. 



Life  Cycle  Events 

See  Section  6. 

Figure  4.  Selected  Subject  Index  (cont'd) 
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SELECTED  SUBJECT 


GUIDEBOOK 


TEXT  REFERENCE 


Test/Support  Tools 

Verification 

4.1 .2.4,  Appendix  A. 

Software  Maintenance 

1.3.2. 

Software  Quality  Assurance 

3.4.10. 

Software  Development  & 
Maintenance  Facilities 

4.3.1  thru  4.3.1 .9, 
Appendix  C. 

Facility  (Support/Develop- 
ment/Maintenance)  Planning 

Contracting  for  Software 
Acquisition 

2.2. 

Software  Development  & 
Maintenance  Facilities 

See  Entire  Volume. 

Coding  Standards/ 
Conventions 

Monitoring  & Reporting 
Software  Development  Status 

See  Appendix  III 
(5  thru  5.2.5). 

Operational  Test/ 

Evaluation 

Monitoring  & Reporting 
Software  Development  Status 

See  Figure  1 and  2.2.1 
(Page  30). 

Verification 

1.2,  1.2.3,  Figure  3. 

Validation  & Certification 

1.2,  1.2.1,  12.2., 
Section  5. 

Software  Development  & 
Maintenance  Facilities 

2.2. 

Life  Cycle  Events 

4.4.3,  8.3. 

Software  Maintenance 

Software  Maintenance 

See  Entire  Volume. 

Software  Development  and 
Maintenance  Facilities 

4. 3. 1.9. 

Figure  4.  Selected  Subject  Index  (cont'd) 
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4.1  INTRODUCTION 


As  the  guidebooks  were  prepared,  several  issues  arose  which  were  beyond  the 
original  intent  and  scope  of  the  series.  These  issues  are  briefly  presented 
in  this  section  with  the  hope  that  they  will  be  considered  for  incorporation 
in  revised  or  consolidated  guidebooks. 


4.2  THE  SYSTEM  AND  SOFTWARE  ENGINEERING  PROCESSES 

Although  these  topics  were  partially  addressed  in  several  of  the  guidebooks 
(i.e..  Life  Cycle  Events,  Quality  Assurance,  Verification)  they  should  be 
more  thoroughly  covered  in  a single  guidebook.  The  systems  and  software 
engineering  activities  which  lead  to  the  System  Specification  and  to  the 
Development  (Part  I)  Specification  are  critical  to  the  success  of  all  software- 
embedded  systems.  Also,  a thorough  discussion  of  the  systems  and  software 
engineering  activities  can  provide  detailed  guidance  to  be  mapped  and  evalu- 
ated against  the  requirements  of  MIL-STD  490.  Special  attention  should  be 
given  to  identifying,  evaluating,  and  minimizing  risks  in  the  development  of 
software.  However,  emphasis  must  be  placed  upon  the  identification  and 
specification  of  system  requirements,  and  the  temptation  toward  early  detail- 
ing of  software  requirements  without  proper  system  definition  should  be 
avoided. 


4.3  DIFFERING  PROCUREMENT  STRATEGIES 

More  research,  analysis,  and  guidance  is  needed  to  help  the  P0  determine  the 
most  appropriate  procurement  strategies  for  any  given  program.  Topics 
requiring  further  discussion  include: 

• What  types  of  contracts  (Fixed  Price,  Cost  Plus)  are  most 
appropriate  for  differing  types  of  procurements? 

• Under  what  conditions  is  it  beneficial  to  the  Government  to 
contract  with  two  or  more  competitive  organizations?  When  is 
such  an  arrangement  most  likely  to  be  unnecessary? 

• How  should  the  P0  contract  for  sufficient  visibility  throughout  the 
System  Acquisition  Cycle? 

• What  are  the  most  appropriate  activities  to  request  from  a Verifi- 
cation and  Validation  (V  and  V)  contractor?  How  should  V&V 
proposals  be  evaluated? 
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4 . 4  SOFTWARE  ASPECTS  OF  THE  SYSTEM  SPECIFICATION 

The  Requirements  Specification  guidebook  focuses  primarily  upon  the  CPCI 
Development  (Part  I)  Specification.  Further  guidance  is  needed  regarding 
the  software  aspects  of  the  System  Specification.  Currently  some  agencies 
are  placing  added  emphasis  on  software  requirements  as  part  of  the  System 
Specification;  not  only  performance  requirements,  but  upon  requirements  for 
standards  and  for  maintainability  as  well.  To  what  extent  does  this 
emphasis  benefit  the  Government,  and  to  what  extent  will  added  costs  or 
added  risks  likely  ensue?  These  issues  should  be  addressed  in  conjunction 
with  the  issues  described  in  4.? 


4.5  THE  BUDGETING  AND  SCHEDULING  DILEMMA 

Although  the  Software  Cost  Estimation  and  Measurement  guidebook  covers  current 
practices  and  discusses  state  of  the  art  techniques,  it  does  not  adequately 
address  one  of  the  most  difficult  orohlems:  budgets  and  schedules  for  most 
software  embedded  C3  systems  are  originally  generated  early  in  the  system 
acquisition  cycle  often  before  the  System  Specification  is  available.  How- 
ever, successful  cost  estimation  and  scheduling  depends  upon  an  accurate 
definition  of  requirements  to  a fine  level  of  detail.  Approaches  to  this 
dilemma  require  the  development  of  a clear  and  agreed  upon  statement  of  the 
problem  followed  by  the  development  and  evaluation  of  several  alternatives. 
This  problem  should  be  addressed  by  a team  containing  personnel  with  a 
thorough  understanding  of  procurement  policy  and  strategy,  system  and  software 
acquisition  management,  software  technology,  and  software  configuration  manage 
ment. 


4 . 6  FIRMWARE  ACQUISITION  MANAGEMENT 
3 

This  C guidebook  series  is  notably  lacking  in  specific  guidance  about  the 
acquisition  of  firmware.  However,  paragraph  3,  m,  (8)  of  AFSC  Supplement  1 to 
Volume  I of  AFR  800-14  states  that  "computer  firmware  will  be  managed  as 
Computer  Program  Configuration  Items..." 


4.7  AN  AIR  FORCE  GLOSSARY  OF  TERMS 

Although  each  guidebook  contains  a glossary  of  the  terms  used  in  that  guide- 
book, It  is  evident  that  many  software-related  terms  are  net  consistently 
used  throughout  the  guidebook  series,  throughout  the  RSS,  or  throughout  the 
Air  Force.  For  example,  the  words  software,  verification,  and  validation  are 
defined  differently  from  publ ication-to-publ ication.  Reliability  and  maintain- 
ability mean  one  thing  for  hardware,  another  for  software.  The  solution  to 
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this  problem  requires  the  adoption  of  standard  and  agreed  upon  terminology  by 
both  software  acquisition  management  and  technical  personnel. 


4.8  VALIDATING  THE  GUIDEBOOK  SERIES 

Now  that  the  guidebooks  have  been  written,  they  must  be  validated  through  use 
j by  the  POs.  Questions  such  as  the  following  should  be  addressed: 

• To  what  extent  are  the  guidebooks  useful  for  new  PO  personnel, 
for  experienced  PO  personnel?  How  can  they  be  made  more  useful? 

• To  what  extent  should  the  guidance  be  expanded  or  elaborated  upon, 
condensed,  corrected,  or  deleted? 


4.9  APPLYING  THE  GUIDEBOOKS  TO  OTHER  POD  AGENCIES 

This  series  was  sponsored  by  ESD  and  is  primarily  directed  toward  ESD  POs. 
However,  much  of  the  guidance  should  be  applicable  throughout  the  Air  Force 
for  800-series,  software-embedded  system  procurements.  ESD/TOI*  will  appre- 
ciate feedback  from  other  DoD  agencies,  as  well  as  from  industry,  regarding 
their  usefulness  and  suitability. 


♦Formerly  MCI . 
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(RETYPED) 

DEPARTMENT  OF  THE  AIR  FORCE 

HEADQUARTERS  AIR  FORCE  CONTRACT  MANAGEMENT  DIVISION.  AFSC 
KIRTLAND  AIR  FORCE  BASE.  NEW  MEXICO  87117 

REPLY  TO  EN 
ATTN  OF 

subject  AFCMD  Computer  Software  Policy  ^7 


to  AFSC/XRF 

1.  The  AFCMD  mission  is  three-fold:  Perform  the  contract  management  functions 
at  our  assigned  contractor  plants  per  ASPR  1-406;  provide  support  to  program 
managers  and  buying  agencies  in  accordance  with  Memoranda  of  Agreement  (MOA) 

or  Letters  of  Delegation*  per  ASPR  20-703.3  and  AFSC  Supplement  thereto;  and 
evaluate  the  contractors'  management  systems  via  our  Contractor  Management 
System  Evaluation  /(CMSE)  Program. 

i 

2.  All  three  mission  responsibilities,  from  their  inception,  have  been 
interpreted  by  mast  concerned  parties  as  pertaining  primarily  to  hardware. 
Acquisition  of  computer  software  relative  to  weapon  systems  (computer  programs, 
computer  data,  and  associated  documentation)  has  been  recognized  as  a major 
concern  within  the  Air  Force  since  1972  and  was  given  DOD  emphasis  in  1976 
with  the  publication  of  DOD  5000.29,  "Management  of  Computer  Resources  in 
Major  Defense  Systems." 

. The  DOD  Directive  implies  the  Services  should  treat  computer  software 
with  the  same  degree  of  importance  as  they  do  hardware.  This  emphasis  on 
software  changes  our  AFCMD  engineering  workload  significantly.  Therefore, 
for  the  present  and  in  the  immediate  future,  AFCMD  must  adopt  a policy  com- 
patible with  our  manpower  levels  and  the  shortage  of  technically  knowledge- 
able manpower  resources  in  the  area  of  computer  software.  Eleven  of  our  twenty 
AFPROs  administer  contracts  with  significant  computer  software  requirements, 
but  we  have  very  few  personnel  who  can  perform  their  ASPR  1 -406(c)  engineering 
functions  relative  to  computer  software  with  the  same  level  of  expertise  that 
they  currently  do  with  respect  to  hardware. 

4.  Due  to  manpower  limitations  and  skill  shortage,  AFCMD  can  only  provide 
surveillance  of  in-plant  computer  software  as  described  below: 

i 

a . Engineering  Surveillance  of  Computer  Software  Accomplished  VIA 

the  CMSE  Program.  The  AFPR0  Engineering  Divisions  now  provide  surveillance 
of  computer  software  relative  to  ASPR  1 -406(c)  engineering  functions  32,  34, 

35,  38,  41,  the  engineering  change  system  portion  of  42,  and  function  43  via 
the  CMSE  Program.  Function  39,  monitor  contractor  value  engineering  programs, 
will  be  incorporated  into  the  CMSE  Program  in  the  next  revision  of  AFCMDR 
178-1,  "Contractor  Management  System  Evaluation  Program." 

b.  Engineering  Surveillance  of  Computer  Software  Not  Accomplished  VIA 
the  CMSE  Program.  The  remainder  of  the  ASPR  1 -406(c)  engineering  functions 
(33,  36,  37,  40,  all  of  42  except  the  engineering  change  system  mentioned 
above,  and  functions  44,  45  and  46)  cannot  be  accomplished  relative  to  computer 
software,  via  the  CMSE  Program.  Concerning  these  ASPR  functions,  AFPR0 
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Engineering  Divisions  will  perform  the  related  surveillance  concerning 
deliverable,  nondeliverable,  and  support  software  only  to  the  extent  definitized 
in  the  applicable  MOA.  This  means  that  for  each  SAR/PAR/CAR  program  with 
computer  software  efforts  required  of  the  contractor,  the  program  office  and 
the  AFPRO  must  specifically  negotiate  into  the  MOA  for  that  program,  all 
computer  software  tasks  to  be  performed  by  the  AFPRO  Engineering  Division  and 
which  (if  any)  of  these  ASPR  functions  are  to  be  performed  by  the  Program 
Office,  third  party  contractors,  or  in-house  laboratories. 

c.  Procurement  Quality  Assurance.  ASPR  1 -406(c)  function  47  (perform 
procurement  quality  assurance)  comprises  a multifaceted  responsibility  relative 
to  deliverable,  nondeliverable,  and  support  computer  software.  This  responsi- 
bility will  be  accomplished  by  review  and  evaluation  of  the  contractor's 
quality  assurance  management  systems  via  the  CMSE  Program,  verification  of  the 
quality  assurance  requirements  of  the  contract  via  AFCMDR  74-1,  and  performance 
of  specific  computer  software  quality  assurance  tasks  delineated  in  the  MOA. 

d.  Other  ASPR  1-406  Contract  Management  Functions.  The  AFPROs  will 
perform  the  remaining  ASPR  1 -406(c)  functions  (1  through  31,  and  48  through  69) 
for  contracts  involving  computer  software  as  they  currently  accomplish  these 
functions  for  hardware.  The  nature  of  these  ASPR  functions  does  not  usually 
require  special  procedures  for  the  computer  software  aspects  of  the  contracts. 

5.  A key  ingredient  to  successful  contract  management  is  the  precise  under- 
standing of  respective  roles  and  responsibilities  by  program  offices  and 
AFPROs.  The  MOA  is  the  vehicle  for  this.  It  is  a logical  approach  for 
delineating  responsibilities  and  ensures  that  AFPRO  support  is  responsive  to 
specific  program  office  needs. 

6.  The  requirement  for  MOAs  is  particularly  important  in  the  area  of  computer 
software  tasks.  Program  offices  and  AFPROs  should  negotiate,  in  detail,  via 
the  MOA,  computer  software  tasks  to  be  accomplished  by  both  the  AFPRO 
Engineering  and  Quality  Assurance  Divisions  because  they  entail  the  largest 
computer  software  roles  in  the  AFPROs. 

7.  We  believe  the  adoption  of  this  policy  is  the  best  way  to  accomplish 
Air  Force  requirements  without  requiring  a substantial  increase  in  AFCMD 
manpower.  Therefore,  we  request  your  dissemination  of  this  information  to 
the  AFSC  product  divisions. 

ORIGINAL  SIGNED  BY: 

MERTON  W.  BAKER 
BRIGADIER  GENERAL,  USAF 
COMMANDER 
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